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METHOD AND APPARATUS FOR ANALYZING DIGITAL VIDEO 
USING MULTI-FORMAT DISPLAY 

FIELD OF THE INVENTION 

[0001] The invention relates to test equipment apparatus and methods for use in 
analyzing digital image data. The data can represent successive frames in a live or 
recorded video program and frames, portions of frames and individual pixels can be 
captured and presented. 

PRIOR ART 

[0002] Certain known video test devices are capable of generating several 
different sorts of test images, and can present one of plural selectable test images on a 
display screen, or two or more of a set of selectable test images, at the same time. 
[0003] A simultaneous display may include a video picture image presented in a 
window area less than the full area of the screen, and a simultaneously displayed graph 
of information that relates to the same video picture. For example, a reduced-size image 
of the picture may be presented in one quadrant, while simultaneously showing in 
another quadrant a graph of composite video signal voltage amplitude versus time, thus 
plotting amplitude over a horizontal line in the video picture raster. Alternatively, a 
vectorscope polar display (saturation versus hue phase angle), can be displayed 
together with the picture data that the vectorscope display represents. Other 
simultaneous data presentations are also possible. Instead of display in quadrants, it is 
also possible to overlay displays, such as by superimposing a graph on top of the 
picture image that the graph represents. 

[0004] Such apparatus are useful in part because the display presents the same 
video picture in two different formats that can be compared and consulted for different 
sorts of information. The display might contain two different graphic presentations, such 
as a plot of composite amplitude versus time and a plot of saturation versus hue angle. 
It can be helpful if the picture itself is presented as one of the formats, i.e., the picture 
appearing together with one or more graphic plots of data derived from the signal used 
to generate the picture. 
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[0005] In some situations, it may be desirable simultaneously to display two or 
more unrelated pictures, for example when preparing to switch an output from one video 
source to another. As another possibility, a picture display may be displayed 
simultaneously with information that is related to the program but not specifically to the 
picture, for example by providing a graphic plot of audio data together with a picture. 
[0006] The present invention relates to particular techniques for displaying 
alternative types of data on a digital picture. Examples of arrangements with 
simultaneous video-related graphic and picture displays are disclosed in US Patents. 
5,166,791 - Crawford; 6,069,607 - Everett et al.; and 6,532,024 - Everett et al., which 
are hereby incorporated. 

[0007] In the Crawford 791 patent, a picture is stored and by sampling is read 
back out synchronously with a portion of an analog raster display, thereby re-sizing the 
image and placing the image only on a selected part of the full raster frame. The 
remainder of the picture area is available for other display information, particularly 
including a graphic presentation of data from the picture, such as a composite amplitude 
versus time plot from a selected horizontal line that can be highlighted on the re-sized 
picture display. The Everett '607 and '024 patents teach generating a digital pixel image 
of the picture, which can be displayed selectively. One or more picture images and any 
graphic plots can be displayed in a particular area of a display screen by writing pixel 
data into a corresponding part of an image memory and generating the display from the 
image memory. The image data can be captured, stored, processed and displayed 
using standard computer processors, display drivers and displays, or using more 
customized but similarly equipped apparatus. 

[0008] It is advantageous to process image data digitally because various 
computational and image processing steps are conveniently accomplished on numeric 
pixel data. If a video signal is available in an analog form such as composite video, it is 
possible to sample the signal or a decoded version of the signal. Samples at regular 
intervals can provide numeric values of luminance/saturation/hue (or another color- 
space encoded set of values) for an array of discrete minimum-sized points in the 
picture, known as pixels (or "picture elements"). This collection of pixel data is a digital 



- 2 - 



SN 10/814,401 - Substitute Specification 
(changes entered) 



version of the picture, comprising an array of minimum area squares or dot-like spaces 
that each occupy a position adjacent to other spaces and together make up the picture. 
[0009] Pixel data encodes three values for each pixel, three values being needed 
to specify a unique color state. The values for a pixel can be converted from one digital 
or numeric color space format to another. The data can be used to generate a larger 
number of pixels by interpolation, i.e., by inserting additional values between values in 
the array. Likewise, the number of pixels can be reduced, for example by replacing the 
value of two or more adjacent pixels with an average or other derived value. The data 
can be converted back into a composite or analog format. It can be mathematically 
processed and manipulated by data processors and/or other circuits. The digital data 
can also be transmitted, stored, compressed, subdivided, threshold detected or 
otherwise analyzed, and generally used in any number of different ways. Among other 
uses, the data can be used to generate a new signal to be displayed, wherein the 
previous signal may occupy a discrete window such as a quadrant of a picture screen. 
[0010] Digitizing by repetitively sampling a signal value and providing a 
succession of data values inherently produces a set of pixel values. There are particular 
standards usefully applied to digitizing picture data, for representing digital video during 
distribution and processing. One type of standard employs digital values for luminance 
(Y) and color difference (Cb and Cr). There are other similar sets of three values that 
can be used to define the color, such as the respective amplitudes of the R, B and G 
color components, or values of luminance, saturation and hue, etc. There are well 
known equations for converting values in one color space to values in another color 
space. Conversions between color spaces are common. For example, luminance and 
color difference might be an advantageous format in which to transmit and store data 
values efficiently. However, R, B and G are useful when referring to drive signals in the 
particular color portions of a circuit. 

[001 1 ] A standard distribution format for YCbCr luminance and color difference 
data takes advantage of the fact that human perception is more acute as to luminance 
than color. The known 4:2:2 sampling and transmission format uses a sampling 
technique wherein the successive samples alternate between color difference Cb and 
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color difference Cr, but each sample has a luminance value. In that way, there are twice 
as many luminance samples as there are values of either Cb or Cr. Stated another way, 
the luminance values can change twice as quickly as the color values. The samples 
alternate: Y1-Cb1 . . . Y2-Cr1 . . . Y3-Cb2 . . . Y4-Cr2, and so forth. It is necessary to 
have a value for each of Y, Cb and Cr to characterize a color state. Therefore, each 
discrete pixel position in the picture needs to draw part of its data from two samples. 
Nevertheless, both the idea of a pixel and the idea of a video data sample refer to the 
minimum encoded or displayed version of the picture, even though a sample and a pixel 
may not always correspond to one another in an exact technical sense. 
[0012] In various situations, it is advantageous for test equipment used in video 
production to be sensitive to different aspects of a picture or a picture signal. Some 
important objectives for a test equipment manufacturer include making test apparatus 
highly versatile so that there are many ways that video data can be presented for 
analysis. It is also important to make the apparatus convenient, so that the operator can 
readily change the presentation of video data from one configuration to another. 
[001 3] It would require an extensive discussion to address all the ways that a 
user of a video test apparatus might advantageously use the apparatus to analyze a 
video signal. Some of the approaches might relate to aspects of the content of the video 
and how the image data was collected. There can be issues respecting encoding for 
transmission and decoding, such as errors that arise in compression, transmission, and 
color space conversions. There are issues as to correct synchronization and timing. 
There may be conditions in which an output device is out of adjustment and produces a 
visible anomaly but the underlying data is legal and should not produce such an 
anomaly. An advantageous video test apparatus should be applicable to checking and 
analyzing video in all of these situations. 

[0014] On the data collection side, it is possible, for example, that isolated pixels 
in a picture are erroneous or out of range. It would be advantageous to facilitate 
analysis of selected pixel positions in a display so as to allow the spatial area of the 
error to be viewed. A challenge is presented to provide a test apparatus that allows 
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attention to be placed on a small area within a larger picture, that does not detract from 
the ability to monitor the larger picture. 

[0015] Errors sometimes are fleeting. In a video signal that is changing many 
times per second, an error in a particular frame can be difficult to find. It would be 
advantageous to facilitate analysis at particular moments in time. A challenge is also 
presented to permit attention to be focused on an instant in time, even while the video 
signal may be advancing rapidly from frame to frame. 

[001 6] Errors or other subjects of analysis may present numeric issues or visual 
issues. Due to the particular conversions between color space definitions, for example, 
it is possible for a three-variable color definition to be a legal color state within accepted 
limits in an original color space, but when numerically converted to a second color 
space to produce values that are out of range for at least one of the variables in such 
second color space. An out-of-range set of three values in a color space is known as a 
gamut error. It would be advantageous to provide a test apparatus that has good facility 
to deal with numeric data and with visual data. 

SUMMARY OF THE INVENTION 
[001 7] An object of the invention is to mediate the foregoing conflicting interests 
in a digital video test apparatus. That is, the invention advantageously serves important 
interests in versatility versus convenience, focus on spatial detail versus a larger view, 
time capture versus flow, and numeric versus visual presentations. Preferably, this is 
accomplished using digital data processing, display driver and display presentation 
apparatus with certain automatic and certain user selection modes, including the 
selection of particular outputs to be presented at a given time. The result is a device that 
is versatile, convenient and highly useful in video production environments and the like. 
[001 8] A multi-format display device for video production and testing is provided 
that generates and selectively maps into an output at least three distinct presentations. 
These presentations can be located selectively at different areas of a display, made 
relatively larger or smaller, selected individually or in pairs or trios, etc., so as to 
facilitate analysis of video data for discrete areas within the video picture or for specific 
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technical aspects of the picture or picture signal. Advantageously, the user can 
selectively devote the full display apparatus to one presentation, or present two related 
or unrelated presentations along side one another, or present several presentations at 
the same time, each preferably representing a distinct way in which the picture, and/or 
the underlying picture data, are viewable for analysis. These distinct ways include the 
full picture, preferably with a movable cursor pointing to a selected area, a zoomed area 
around a chosen area (pointed out using the cursor or selected by another technique 
such as threshold selection criteria), and a tabular numeric analysis of the samples. 
Advantageously, the tabular numeric display includes swatches demonstrating the 
visible results the sample values for one or more pixels and/or samples at the position 
of a cursor shown in the complete picture and in the zoomed area. 
[001 9] A video processor and a controller responsive to user input selections 
display the complete picture from the input signal on a limited part of the device display 
area, such as a quadrant of a digitally controlled standard display. The complete picture 
can be a resized version of the data contained in the complete picture, adjusted 
mathematically to reduce the spatial resolution, or adjusted by sampling the pixels 
and/or sample values as necessary to cause the full image area to occupy such limited 
part of the display device. 

[0020] Selectively, preferably using another limited part mapped into the display 
area, a picture zoom is presented. The picture zoom area shows a visual enlargement 
of a subpart of the picture at and around a cursor position. The controller and the video 
processor can operate automatically and/or in accordance with user input to select one 
or more cursor positions. For example, a cursor position can be selected automatically 
to show the area at and adjacent to an illegal pixel or sample value, a color gamut 
violation, a specific data value, etc. 

[0021] The zoom area is preferably large enough that individual pixels or data 
samples are made readily visible as blocks, due to the visual enlargement of the data 
contained either in the original input signal or in the resized version of the complete 
picture that is displayed in the first limited part of the display area discussed above. The 
zoom area also shows the cursor or otherwise identifies particular pixels that are 
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subjected to scrutiny. The picture pixel or sample data is mapped into blocks of pixels in 
the display device output. 

[0022] The video processor and controller generate and selectively present a 
tabular form of specific pixel numeric data for one or more pixels at the cursor, for 
example by mapping this presentation into another part of the display screen or 
permitting user selection of this presentation in lieu of (or in addition to) the other 
available displays. One or more of the optional selections preferably contains the full 
picture, the picture zoom limited to an area of scrutiny around the cursor, and a tabular 
data presentation wherein a color swatch is shown in a color state corresponding to the 
pixel data values. Furthermore, these optional selections can be shown individually or in 
groups of two or more, mapped to all or to a discrete part of the output display area by 
operation of the video processor and controller. 

[0023] Graphic plots of picture data, audio, time code, and other associated 
information can be placed selectively in remaining areas or shown by default. The 
controller and video processor are operable to accept pixel selection criteria, to identify 
pixels in the display that meet the criteria, and to scrutinize the selected pixels in 
sequence, subject to manual selection override. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0024] A number of additional objects and aspects are apparent from the 
appended description and the associated illustrations of preferred embodiments, 
wherein: 

[0025] Fig. 1 is a block diagram showing the primary functional elements of a 
multi-format display according to the invention for analysis of digital or pixel images. 
[0026] Fig. 2 is flowchart showing steps involved in mediation between automatic 
and manual selection of display areas to be subjected to scrutiny. 
[0027] Fig. 3 is a display image capture showing the invention as used to analyze 
a color bar test pattern that has been re-sized. 



-7- 



SN 10/814,401 - Substitute Specification 
(changes entered) 



[0028] Fig. 4 is a display image substantially as shown in Fig. 1 , for illustrating 
the automatic selection of areas for application of scrutiny, and with alternative graphic 
plots being selected. 

DETAILED DESCRIPTION 

[0029] Referring to the block diagram in Fig. 1 , a multi-format display device and 
associated method are shown, particularly for application to video production and test 
uses. The device and method are arranged to facilitate the analysis of a stored or active 
(e.g., live) video signal, using different information display formats at different areas of a 
display. According to the invention these areas display a version of the incoming video 
input signal, a relocatable zoom area from the video input signal, associated with a 
cursor position, a tabular information display showing digital sample values that 
contributed to the video at the cursor as well as corresponding color swatches, and can 
further include certain graphic plots representing time, amplitude, and color-space value 
plots that demonstrate aspects of the video input signal. 

[0030] In Fig. 1 , the video input signal 22 is represented by an image box. The 
invention is applicable to a digital video signal, but the video input signal 22 can be 
supplied initially as a digital progression of sample values, or can be digitized by a 
conventional analog/digital converter (not shown), that can be included as an element of 
the test apparatus. Similarly, the video input signal 22 can be derived from stored 
information such as a compressed video data, decrypted data, etc. In any event, the 
incoming video input signal 22 is provided in or converted to a digital format supplied to 
or generated in the device. 

[0031 ] The digital values of the incoming signal can be in any of various forms, of 
which several conventional forms are known. Thus, the samples can be carried in a 
parallel or serial communication path. The incoming signal and the digitized samples 
can assume or be converted among any of various formats, including custom or 
standardized formats such as according to NTSC, PAL, CCIT, SMPTE, HDTV, etc. The 
signals can be conventional serial digital interface (SDI) signals as used in broadcasting 
applications or developed by decoding or decompressing stored data files such as 
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MPEG data or the like. The invention is generally applicable to these and other such 
variations, wherein at some point in the processing of the video signal, a digitized 
version of the signal is available or can be generated by the test device itself. 
[0032] In addition to the video input signal 22, signals developed from user 
controls 25 and handled by a controller 28 are coupled to a video processor 30 that 
generally comprises input/output circuits, a memory sufficient to store digital data 
representing all or part of the incoming video input signal 22, and a display generator 
section that produces from the video input signal 22 and signals such as switch settings 
from the user controls 25, a formatted display 32, on a display device. In the example, 
shown in Fig. 1 , the formatted display 32 comprises several discrete zones of a single 
integral display field, and is produced by a display generator 34 associated with the 
video processor 30. 

[0033] The formatted display 32 can be produced in several ways, including 
coupling the video processor 30 and display generator 34 to generate separate and 
different displays on different display devices. However according to the preferred 
embodiment shown, the display generator 34 is arranged to allot separate zones in the 
display area of a single display device to represent several different display format 
types. This preferably is accomplished by the display generator 34 composing a pixel 
image of values for the formatted display 32 that can be read out through a standard 
digital monitor device. For example, the display generator 34 can provide frame data for 
a digital display driver operable to produce a VGA, XVGA or other standard format 
display, or can drive a display device, e.g., by producing RGB signals, composite 
signals to be decoded, or otherwise to encode display information to be converted into a 
visual presentation at another point along a signal path, e.g., by a remote device. 
[0034] In the preferred arrangement, the formatted display 32 is shown on a 
single digital display device and the display generator 34 produces a multi-format image 
by composing a frame of pixel data values to be displayed. The composed data to be 
displayed can be a single frame that demonstrates aspects of the video input signal 22 
at a point in time such as a single frame or field. Preferably in at least some modes of 
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operation, the display is composed repetitively, being thereby actively changing in 
response to changes in the character of the video input signal 22. 
[0035] The apparatus including the video processor 30 incorporates or is coupled 
to at least one device operable to present the formatted display 32 of information at 
least partly from the video input signal. For this purpose, the apparatus includes a 
controller 28 that can be coupled to or between the video processor 30 and to at least 
one control input 25 by which certain control selections are made, preferably by a user 
who manipulates the controls to change the display from time to time. The controller is 
operable to control a nature of the information displayed by the video processor in 
various ways, including selectively to display information regarding the video input 
signal in alternative ways, to selectively position a cursor for pointing to an area of the 
display to be subjected to analysis, to select certain graphic plots and to enter data that 
will be used in composing the particulars of the display. 

[0036] The video processor 30 is operable to produce, on at least part of an area 
of the display device, a visual display that corresponds to the span of the full picture 40 
of the input video signal. As shown in Fig. 1, a frame of the full picture is placed in a 
limited area of the full formatted display 32, namely in an upper quadrant in the example 
shown. The full display area of the formatted display 32 preferably is presented on a 
display or monitor device that is at least capable of displaying a picture to the maximum 
resolution expected in the input video signal over the full area of the screen. In other 
words, the display is capable of producing a number of distinct horizontal lines each 
comprising a row of laterally spaced pixels, such that the incoming picture comprises a 
given number of horizontal lines of which each line has a given number of lateral pixels 
or samples. The number of total pixels is the product of said given numbers . In order to 
display the same picture information on this display device, for example in one quarter 
of the area of the full screen, it may be necessary either to attend to a change in 
resolution, or perhaps to employ a difference in timing or sampling. 
[0037] According to one embodiment, the video input signal 22 can contain a 
raster scan of horizontal lines. In that case, a stream of digital samples can be provided 
to represent the changing pixel color values across each successive line (luminance 
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and color difference YCbCr or color component amplitude RBG, or luminance- 
saturation-hue, YUV, etc.). The samples can be received in analog composite or 
component format in the incoming signal or can be digitized from composite or 
component signals such that the sampling rate defines each minimum increment or 
pixel as one sample value. Alternatively, the video input signal 22 can be previously 
digitized and supplied serially or in a serial/parallel stream or as a data file, that contains 
or is decoded to provide an array that encompasses a given number of pixels . In order 
to present the full incoming picture contained in the video input signal 22 in one quarter 
of the display, the signal can be stored and read out to correspond to portions of the 
overall display. 

[0038] There are several ways in which resizing can be accomplished. For 
example, if the output display device used to display the formatted display 32 as an 
output frame has a resolution that is high enough, e.g., at least four times the resolution 
of the incoming picture, then all the pixel values in the incoming picture can be read out 
into the area corresponding to the quadrant or other portion in which the full picture 40 
is to be represented in the output. Usually, however, incoming picture signals such as 
the picture contained in video input signal 22 are encoded and decoded at a resolution 
that is comparable to the maximum resolution of the display device, so as to take full 
advantage of the possible detail and picture quality. 

[0039] In the event that the formatted display 32 is composed of a pixel field and 
the incoming video input signal 22 is composed of a pixel field, then the full field of 
incoming pixels can be mapped spatially to the pixels in the quadrant where the full 
picture 40 is displayed. This will normally involve converting the number of pixels in the 
video input signal 22 to the number in used for the quadrant in the output. For example, 
assuming equal total resolution (i.e., the same counts of pixels in length and width 
encoded in the digital input and provided by the output device), the output signal can be 
a decimated version of the input signal wherein for a one-quadrant presentation, one of 
four pixels in each horizontal line is used and every fourth vertical line is used, these 
pixels mapping into one quadrant of the output display. (A different proportion can be 
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used if the output is to occupy some proportion other than one fourth of the available 
display field.) 

[0040] Rather than decimating, the pixel values can be processed 
mathematically. For example, every four adjacent pixels in the input signal can be 
averaged as to their color defining variable values, and the average used as a pixel 
value in a one fourth sized field, namely a quadrant of the area of the display device . 
This amounts to reducing the pixel-by-pixel resolution of the video input signal 22 in 
order to present it in a reduced area of a display having a resolution equal to the 
resolution of the original input signal. Of course, there are other possibilities. For 
example, if the input signal has a coarser resolution (a smaller number of total pixels) 
than the display device, then less decimation or mathematical averaging may be 
necessary. 

[0041] The matter of pixels and sampling is analogous to the amplitude of a 
composite signal that is scanned in an interlaced or non-interlaced raster pattern in an 
analog sense. For such a display, every other field of an interlaced signal could be 
stored line-by-line and read out during scanning of the quadrant area shown. 
[0042] In a hybrid arrangement, it is further possible to receive incoming video 
input signal 22 in digital form, to convert the digital data to an analog signal, and to 
resample the analog signal thus obtained to provide pixel sample data that can be 
processed as described, namely displayed in a limited area, such as one quadrant, of 
the multi-format display output. 

[0043] In a preferred embodiment, the video input signal 22 is a serial digital 
signal having a bit rate or resolution equal to the full resolution of which the display 
device is capable. The incoming pixel data values are mathematically processed in a 
running average and decimated to reduce the number of pixels to a number that maps 
into the output display quadrant. 

[0044] The foregoing mapping of the complete input signal into a limited area of 
the output display field is only part of the composition of the output display field. 
However, this aspect causes the full signal to be represented in the display of the test 
apparatus of the invention, in close proximity to other display portions that are arranged 
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to apply or concentrate analysis and attention on specific portions and/or aspects of the 
signal. 

[0045] According to another aspect of the invention, a specific spatial portion of 
the compete signal can be identified with a cursor 42. The cursor is movably positioned 
by means of the user controls 25 and/or by operation of the controller 28. The cursor is 
generated by the video processor and is superimposed on the signal represented in the 
area of the full picture 40 of the output formatted display 32. 

[0046] According to a further aspect of the invention, an area 44 of the picture 
contained in the video input signal 22 is zoomed or enlarged and placed in a first 
supplemental display area that preferably is another part of the display area available in 
the formatted display 32. This picture zoom area 44, shown in the upper left quadrant in 
the formatted display 32 of Fig. 1 , is used to display only the subset of the complete 
picture that is located at and adjacent to the cursor 42. The area used for this portion of 
the input signal can be displayed on an area that is of comparable size to the area used 
for the full picture 40 in a quadrant of the area of the formatted display 32, but in any 
event the area shown is substantially enlarged. 

[0047] It would be possible to display the picture zoom area 44 in a first 
supplemental display area at a resolution that is equal to a resolution that would be 
obtained by displaying the video input signal 22 in the full area of the output formatted 
display 32, which would be an enlarged display relative to the complete picture 40 
displayed in the first area, namely the quadrant shown. Preferably, however, the picture 
zoom area 44 is displayed at an even larger relative size (i.e., a lower or coarser 
resolution) so that individual pixels are readily distinguishable as blocks or swatches of 
one color state each, as shown in Fig. 1 . 

[0048] The picture zoom area occupying the first supplemental display area is 
part of the display that is composed by the video processor 30 from the video input 
signal 22. The video processor maps pixels that are at cursor 42 to a block of pixels in 
picture zoom area 44 of the formatted display 32. 

[0049] The cursor 42 identifies one or more pixels or data samples in the image 
defined by the input signal. It is possible to include in the manual controls a capability to 
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select a cursor size, such as a single pixel or two or more adjacent pixels in a line or as 
appearing adjacent to one another is successive lines, or in another arrangement in 
which the pixels are adjacent or otherwise are related. A nominal cursor size is four 
adjacent pixels in one horizontal line. 

[0050] The zoomed picture portion that is used by the video processor in 
mapping an area of the input signal to the supplemental display area is changeable in 
position in the display field (location in the picture) and optionally in size or shape. One 
or more of these aspects is determined by settings via switch inputs or data supplied to 
the controller 28, and at least the location of the cursor in the picture is determined in 
this way. The position can be a manual or automatic selection by positioning or by 
counted pixel coordinates. Using manual selection, the user typically sets and displays 
the zoom image from one cursor location and thereafter moves the cursor to a different 
location and locally zoomed area. It is possible to permit the cursor location to be held in 
a memory and to set a new position while retaining the previous location in memory, 
with a capability to revert to a previous position or to retain the data relative to the 
previous position. 

[0051] The zoomed picture can have a preset number of pixels or can have a 
varying number of pixels. The varying number of pixels can be displayed in a preset 
portion of the area of the formatted display 32 (i.e., in a preset vertical-horizontal span ). 
Alternatively, the relative sizes of the respective windows for the full picture 40 and the 
zoomed picture in picture zoom area 44 can be variable such that enlarging one can 
reduce or encroach on the space for the other. These changes are readily provided 
because as a matter of mapping, the video processor 30 and controller 28 operate in 
conjunction to compose the output image according to user controls 25 and/or 
automatic criteria, and by inserting data into a display memory that is then read out to 
present the output display . The mapping by the video processor 30 can include pixel 
data from the image as described, pixel data at different or alternatively selectable 
locations and resolutions, as well as graphic elements. 

[0052] The pixels encompassed by cursor 42 potentially have different color 
states. Further, the pixels are generally the product of sampling that involves quantifying 
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the respective values applicable to the pixels in a color space, i.e., at least three 
variable values to define a color state. According to an inventive aspect, at least a 
second supplemental display area under the composition control of the video processor 
30, which again is preferably a subdivided part of the same area of the formatted 
display 32, is populated by the video processor with a tabular data report showing the 
video data characteristics at a limited area identified using the cursor and constitutes a 
pixel information area 48. 

[0053] According to another aspect, the multi-format output produced by the 
apparatus of the invention, preferably as composed by the video processor and 
controller 30, contains a report of the color space variable data. This report as shown in 
Fig. 1 can include information that numerically defines the cursor location by the 
horizontal line and serial sample numbers of the pixel(s) encompassed by the cursor, 
and also includes the values of the color space variables. 

[0054] In a preferred embodiment, the serial digital video data can include sub- 
sampled chroma information wherein each pixel corresponds to a sample value. Each 
sample of the video can have an associated luminance value Y and color difference 
value Cb or Cr. In standard 4:2:2 video, each color difference value has an associated 
luminance value, such that the luminance value can change from one sample to another 
but only one color difference value can change with each new sample. As a result, it is 
useful to determine the resulting color. According to the invention, a color swatch 54 is 
associated with each sample or pixel, the color swatch having the same color as shown 
in the picture zoom area 44, but shown in the pixel information area as a separate block 
that is associated with the numeric value of each of the luminance and new color 
difference value that is new to the associated sample. 

[0055] It is possible to display the numeric color difference values in different 
ways. For example, the numeric values can be stated in the YCb-Y'Cr sequence shown 
or in another sampling sequence or set of color space variables. The numeric values 
can be in decimal, hexadecimal, binary, etc. A new color swatch can be displayed for 
each new value of Y or Cb or Cr, instead of once per new sample containing a pair 
(either YCb orYCr). 
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[0056] The preferred tabular report in pixel information area 48 of the invention 
includes a sample number 52, a sample color swatch 54, and sample values shown by 
both a name designation 58 of the variable (Y, Cb, Y', Cr) and its numeric value 62. 
[0057] The apparatus of the invention can be used to process a video input 
embodied by a full rate video source in which the program changes from moment to 
moment. The image that is produced and displayed by the video processor preferably 
varies in real time with the changing picture. In an actively changing video input signal, 
the changing frames can cause the data displayed to change quickly, so among the 
manual controls it is possible to set a cycling rate whereby there is some dwell in that 
the display only changes at a predetermined rate, regularly sampling at least one 
picture frame but not changing with every new frame that is received. Alternatively, the 
frame shown can be arranged to be a freeze frame that is selected by manual operation 
of a user control 25, e.g., operating a pushbutton switch or operating other inputs that 
cause the device to freeze and display a frame at a particular time or upon the 
occurrence of particular conditions such as the occurrence of predetermined data 
values or other conditions in the sample data. 

[0058] In order to meet the data processing requirements needed to resize and 
locate the full picture 40, apply the cursor 42 at the required location, generate the zoom 
presentation in picture zoom area 44 and also produce the tabular display at pixel 
information area 48, it is possible to employ a processor, but preferably some of the 
functions are served by using dedicated digital networks such as field programmable 
gate arrays to serve one or more of the display composition and presentation functions. 
For the most part, these functions involve processing the incoming picture of video input 
signal 22 and loading values into a display memory to produce the resized, zoomed and 
tabular versions of the input signal. 

[0059] In the embodiment shown in Fig. 1, there are additional graphic 
presentations made. The specific graphs shown, namely a vectorscope or polar plot 72 
of color saturation as a function of hue phase angle, and a plot 74 such as the signal 
amplitude (which may corresponds to luminance, for example) over a horizontal line, 
can be conventional plots, for example wherein the vectorscope plot 72 presents the 
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saturation/hue relationship over the entire picture . Alternatively, the vectorscope plot 72 
and/or the composite amplitude plot 74 can be made specific to the area of scrutiny 
shown in the picture zoom area 44. Furthermore, the video processor 30 can be 
switched between modes in which such plots are applicable to the whole picture (or 
whole horizontal line) or are applicable only to the zoom area. 
[0060] Such operations can be accomplished within the video processor and 
controller, which preferably include a resizing engine operable to present the input video 
signal on said part of the area of the display device, the resizing engine operating by at 
least one of recalculating values, decimating, sampling and reading out pixel values so 
as to reduce a number of pixel position in the input video signal to a smaller number of 
pixel positions in one display area such as the area used for full picture 40 and also to 
zoom or enlarge the pixel values in another area such as picture zoom area 44. 
[0061] As discussed above, the control inputs 25 can be used manually to 
operate the controller 28 so as to select from the video input signal 22 an area of 
particular scrutiny, among other selections as needed to switch between optional 
display modes. The video processor is operable simultaneously to present the video 
input signal 22 as a full picture 40 on one part of the area of the display device, a zoom 
image for placing scrutiny on a selected portion of the picture by enlargement in a 
supplemental display area such as picture zoom area 44, and by display of tabular data 
with color swatches for providing a further alternative and even more-detailed form of 
data for analyzing the video signal. 

[0062] The location is governed by the location of the cursor 42 on the picture, 
and the cursor can encompass data for at least two successive points representing one 
of pixels and samples from such part of the area of the display device. As discussed, 
the location of the cursor on the picture can be determined by manual inputs. According 
to one aspect, the cursor position can be determined by relative movement controls 
such as a mouse, trackball, joystick or similar control. The cursor position could also be 
entered numerically as a line number and serial sample number in the line. 
[0063] According to another inventive aspect, the video processor is responsive 
to control inputs that define selection criteria whereby one or more locations are 
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selected as a function of the video data values corresponding to the location. The 
controller 28 in conjunction with the video processor 30 automatically selects from the 
input video signal at least one area of particular scrutiny, and automatically selects a 
cursor position, based upon data in the video signal meeting the selection criteria. 
However, this function of automatic selection also is arranged to accommodate manual 
selections. 

[0064] The controller and the video processor need to coordinate between 
automatic and manual selection of the area for particular scrutiny. This coordination is 
shown in Fig. 2, which is a flowchart in which the decision triangles branch between 
automatic and manual selection of one or more cursor positions. The manual selection 
can supersede automatic selection while active. The manual selection can supersede 
automatic selection for a limited period of time. The manual selection can selectively 
determine which one of a plurality of cursor positions is to be active. Certain selectable 
positions can be determined automatically as a function of pixels at the corresponding 
locations meeting selection criteria. Other selectable positions can be manual 
selections. Furthermore, the selection criteria preferably can be adjusted by manual 
control inputs, for example changing selection threshold values or the like. 
[0065] The loop in Fig. 2 can be executed for each display field to be processed. 
The entry point branches at decision block 81 based on whether a current manual 
control input is received. When no manual input is active, the flowchart proceeds to 
block 82 and tests for whether the automatic mode is active. If there is no manual input 
and the mode is not auto, the flowchart loops. If a manual control is input, block 81 
causes the manual location selection to supersede automatic selection. In that case, at 
block 83, the cursor location selected manually is entered for obtaining the picture zoom 
and pixel tabular data is location. 

[0066] If auto mode is active (decision block 82), it is possible that no particular 
sample has been selected at auto-locate decision block 84, in which case the flowchart 
loops as if not in auto mode. According to an inventive aspect, however, selection 
criteria can be entered that cause the location(s) of one or more pixels that meet such 
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criteria to be assumed as the selected cursor location automatically. In that case, block 
85 selects the sample location(s). 

[0067] Whether the cursor location to be used is entered manually or results from 
automatic selection, block 86 provides for one or more of several alternative or 
simultaneous display fields to be presented in the display to be shown. It is possible 
selectively to display the complete picture alone, on the full area of the display, or to 
display the resized version shown in Fig. 1 and to display one or more of the other 
display frames together with the complete picture. In the preferred embodiment, at least 
certain of the display modes include the full picture 40, the zoom picture and the pixel 
information detail . Furthermore, the preferred formatted display also can include one or 
more additional data plots such as plots 72, 74 in Fig. 1 . The actual plots 72, 74 shown 
in Fig. 1 are stylized and represent plots that might appear when the video input signal 
22 contains a color bar test pattern. 

[0068] Fig. 3 shows the invention applied fully to the depiction of a color bar test 
pattern, namely a progression of color bars in order of increasing luminance value. For 
purposes of illustration, cursor 42 is placed at a transition between colors immediately 
adjacent to a contrast between color states of horizontal lines that are over and under 
the cursor 42. As discussed, the full picture 40 is resized from a resolution that might fit 
the full display screen and mapped into the upper right quadrant. The cursor 42 is 
superimposed on the picture, in this case using a dark rectangle with vertical center 
lines. 

[0069] As shown schematically by the arrows, the resized picture is enlarged or 
zoomed in the area of cursor 42, and the particular area around the cursor is displayed 
in picture zoom area 44. For purposes of illustration, in Fig. 3 the picture zoom in picture 
zoom area 44 is generated from the resized full picture 40 as opposed to being 
generated entirely from the incoming original picture signal (such as video input signal 
22 in Fig. 1), which would also be possible. However as shown in Fig. 3, the process of 
resizing the image to one quarter or less of the its original resolution has produced an 
artifact that shows that a local averaging technique has been applied. In the 
embodiment shown in the example of Fig. 3, transitions in color state can be seen to 
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progress in steps from one color bar to the next, each pixel at the transition representing 
a step in the change of color state from one pixel to the next in a horizontal direction. If 
the incoming video input signal 22 had been decimated, the transition between color 
bars could occur abruptly from one pixel to the next at the horizontally abutting edges of 
adjacent color bars, but in this case instead there is a transition showing the results of 
mathematically sampling, averaging or otherwise reducing the resolution in steps. The 
transition in several steps is clearly visible in the picture zoom area 44 in Fig. 2, but is 
not seen in the complete picture. This illustrates one of numerous aspects that might be 
of interest to an engineer or technician using the apparatus of the invention to analyze a 
video signal. 

[0070] The picture zoom area 44 presents a detailed visual presentation of the 
cursor area of the full picture 40, and the pixel information area 48 presents a detailed 
data representation of the pixels within the cursor selection box, as shown by the 
phantom dash-dot arrow in Fig. 3 . The number of pixels in the cursor selection box can 
be variable from one to several (e.g., one or four or eight, etc.) based upon user 
selection as entered via the user controls 25 and used by the controller 28 and video 
processor 30 in the generation of the display. 

[0071] Fig. 4 illustrates another advantageous aspect, namely the use of 
automatic selection criteria to determine one or more cursor locations, and in this case 
wherein a color gamut value violation is at least one of the criteria by which a location 
selection is made. A color gamut violation can occur, for example, when a value for one 
of the three applicable values in a color space definition exceeds the span of permitted 
values. Inasmuch as the values in different color spaces are mathematically related, it is 
possible and sometimes occurs that values are adopted that are legal in one color 
space but illegal in another. In the embodiment shown in Fig. 4, the white burst points 
are assumed to represent pixel values wherein at least one of the associated color 
space values exceeds some threshold value. The threshold value can be a 
predetermined value such as the maximum permitted value a value such as (but not 
limited to) an illegal value of luminance, color difference Cb or Cr, saturation, hue angle, 
R B or G amplitude, etc., that exceed the permitted maximum and minimum according 
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to an industry standard. Alternatively, the applicable threshold can be an arbitrarily 
selected value that the operator chooses to monitor, to find and focus attention on pixels 
having one or more color space values meeting a manually entered threshold value 
criterion. 

[0072] In the embodiment of Fig. 4, the automatically selected pixels 114 (for 
examples pixels that offend some threshold criteria) have associated locations, that in 
the automatic mode shown in Fig. 2 produce sample selections. The operator using a 
manual control "scroll" input (not shown) can advance from the manually selected 
cursor location to one of the locations of automatically selected pixels 114, thereby at 
least temporarily moving the cursor 42 to one of the automatically selected pixels 114, 
and/or advancing from one automatically selected pixel to the next. In so doing, the 
picture zoom display frame is changed to zoom the location of the next automatically 
selected pixel 114, and the associated numeric data for the new pixel is displayed in the 
pixel information area 48. 

[0073] In this way, the video processor is operable responsive to the control input 
to define a selection criteria and automatically to select from the input video signal at 
least one area of particular scrutiny based upon data in the video signal meeting said 
selection criteria. The video processor simultaneously presents the input video signal on 
part of the area of the display device, a zoom image of an area of particular scrutiny at 
and around the selected area on a first supplemental display area of the formatted 
display 32, and a detailed analysis of the offending pixels, and preferably their 
immediate neighboring pixels, in the second supplemental area , these areas preferably 
being different areas of one integral display or presentation. 

[0074] According to the embodiment shown in Fig. 4, one of the display fields is a 
gamut error iris plot 112. This display is substantially as disclosed in US Patent 
Application SN 09/726,093, filed November 29, 2000, entitled Method And Apparatus 
For Polar Display Of Composite And RGB Color Gamut Violation, which is hereby 
incorporated for its teachings of methods and apparatus for detecting and displaying 
gamut violations. The display is a polar plot wherein the angle represents hue, but the 
radius represents a combination of at least two color space values. In one embodiment, 
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the radius is equal to the sum of the color saturation and luminance, which is the extent 
of excursion of the composite value, and has predetermined maximum and minimum 
values that are shown on the exemplary plot as radially spaced circles. Similar graphics 
can be used simultaneously to display R, G and B amplitudes as a function of hue, 
wherein the three variables likewise should plot to points within predetermined limits. 
[0075] According to the invention, the selection criteria that can be selected can 
advantageously include a gamut error violation, and the areas of the display that are not 
used for progressively more detailed scrutiny of the full picture 40, the zoomed cursor 
area pixels in picture zoom area 44 and the cursor pixel tabular data at pixel information 
area 48, can include a gamut error plot 112. 

[0076] Fig. 4 also shows that the plots displayed in conjunction with the picture, 
zoom and pixel data displays need not all be related to the video aspect of the picture. 
For example, the multi-format display can permit the selective display of a audio graphic 
120 for showing the amplitude and or phase attributes of an audio surround sound 
signal embedded in the serial digital program that also includes the video programming 
information. 

[0077] The video data characteristics displayed in the picture zoom area 44 
(namely, the first supplemental area in addition to the area of the full picture 40) and the 
tabular pixel information area 48 (the second supplemental display area) can include 
sample data for at least two successive points representing one of pixels and samples 
from the part of the area of the display device associated with the cursor, whether 
manually or automatically positioned. In the embodiments shown, the samples are 
assumed to be within the cursor at one selected position. It is also possible to show 
tabular data for two or more locations in the picture, such as a number of points having 
gamut issues as shown in Fig. 4 (e.g., the automatically selected pixels 114 
representing a gamut error, or meeting a numeric user selection criteria if not a gamut 
error per se). In the event that locations are selected on multiple horizontal lines, the 
tabular pixel information area 48 can include multiple tables, each table as shown, but 
the tables placed laterally along side one another in the display. Other variations in the 
display positioning can also be permitted, such as selecting which of the display 
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alternatives is to be included and which alternative is to be place in which quadrant or 
other subdivision. It is also possible to superimpose the displays, for example 
superimposing the graphic plots 72, 74, 112, 120 on top of the full picture 40. 
[0078] In the embodiment shown, the video data characteristics displayed in the 
second supplemental display area as numeric values for identifying the successive 
points are numeric values quantifying a video color characteristic of each of the 
successive points, such as 4:2:2 samples or pixels. Each point (each sample or each 
pixel) preferably includes a color swatch, namely a visual presentation of a zone that 
has the video color characteristics that correspond to the color variable data values at 
the sample that is also represented numerically. This aspect provides a very quick and 
convenient way to compare and cross reference between the complete picture cursor 
area, the zoomed picture area and the pixel data area, without the need to count lines 
or pixel positions or otherwise to pore over the displays. 

[0079] As described, the invention is a multi-format monitor for video signal 
analysis comprising video signal analysis and processing apparatus operable to present 
a display having at least three distinct selectable display areas. A first of these display 
areas include a full visual presentation of the video signal in one of said display areas, 
occupying an area less than the full available area of the display. This full visual 
presentation at least selectably includes at least one cursor that identifies a limited part 
of the visual presentation containing at least one pixel defined by one of a sample value 
and a discrete minimum size zone. 

[0080] A second of the display areas has an area zoom presentation in which the 
limited part associated with the cursor is zoomed in size to show a local area including 
and surrounding the pixel. Preferably this area is of sufficient size to readily distinguish 
individual pixels at the resolution and area size used. 

[0081 ] A third of the display areas contains a pixel data zoom containing a 
numerical analysis applied to the pixel. In a preferred embodiment, this numeric data 
includes three color space variable values, although as shown by the foregoing 
examples it is possible and advantageous to represent the data as chroma sub-sampled 
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data wherein there are more sampled occurrences of one variable (Y) than another (Cb, 
Cr). 

[0082] The multi-format monitor further includes a video signal input for providing 
a video signal to be analyzed, the video signal including at least one of successive 
picture frames and fields. This input can be arranged to accept serial digital video, 
stored video files, analog composite or component signals to be digitized in the 
apparatus, etc. 

[0083] A video processor is coupled to the display and is operable to produce a 
display of information at least partly from the video input signal for said display areas. A 
controller is coupled to the video processor and to at least one control input, wherein the 
controller is operable responsive to a user input to control selection of information 
displayed by the video processor, and said selection selectively includes manual 
selection of a position of the cursor, and automatic selection of the position of the 
cursor, based upon user selection criteria. 

[0084] The invention likewise can be considered a method for presenting video 
information for analysis, for application to a video signal. As above, the signal could be 
represented by a stored video program, a presently processed program or another 
similar source. The steps involved include providing a multi-format display having a 
plurality of display areas; displaying the video signal as a complete picture at one of the 
display areas, while preferably providing a cursor for identifying a position in the picture; 
enlarging and displaying at a second of the display areas a portion of the picture 
representing a limited area at and around the position identified by the cursor forming a 
picture image zoom that is substantially enlarged compared to a corresponding area in 
the complete picture; and numerically representing at least one pixel associated with the 
cursor by at least one data value associated with one of a position and an appearance 
of said at least one pixel, and displaying a tabular presentation of the data value at a 
third of the display areas. 

[0085] The limited area at and around the position identified by the cursor forming 
the picture image zoom is enlarged so as to present individual pixels as discrete blocks 
of the multi-format display. Furthermore, a tabular presentation that provides an 
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alternative presentation of the visual representation of discrete blocks is provided, and 
preferably includes in the tabular presentation some color swatch pixel appearances 
that correspond to the data values shown. 

[0086] The method also includes selectively controlling the cursor position that 
the display focuses upon of for analysis. The cursor location on the complete picture or 
frame can be determined manually or automatically or by a combination of manual and 
automatic inputs. A user control input permits entering of at least one of a manually 
selected cursor position on the picture and at least one manually defined data selection 
criterion. After moving the position if necessary or filtering through the picture pixel data 
for occurrences that meet the criteria, the manually selected cursor position and/or 
filtered pixel cursor positions are selectively displayed. That is, concurrently or 
selectively these cursor positions are marked, zoomed and used to populate the pixel 
information tabular displays and swatches. In the complete picture, one cursor position 
or two or more cursor positions can be pointed out and focused upon by presentation in 
the tabular data, preferably being shown and marked on at least one of the complete 
picture and the picture zoom, by highlighting, graphics, target-like graphic forms, etc. for 
identifying the currently selected cursor position. 

[0087] The method of the invention can also include graphically displaying, for 
example at a fourth of the display areas, at least one variable or plot that is associated 
with a data selection criteria by which the automatically selected cursor position is 
determined. In a preferred embodiment, a gamut error type graph can be provided as 
the variable plot, wherein a selection criteria involving two or more variables plotted as a 
function of a third variable is used to plot excursions beyond a predetermined 
acceptable threshold level. 

[0088] Preferably, each of the foregoing display areas produced by a video 
processor and controller composing the contents of the display frame by processing 
contents of an image memory, and generating the display frame from said contents. As 
discussed, this process operates the video processor portion in conjunction with a 
controller responsive to user controls, to resize the complete picture for display in a 
subset of the display frame, to resize a portion of the complete picture around a cursor 
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for zoomed display that shows pixels as distinct areas, and generation of tabular data 
on the pixels at the cursor together with any necessary or desirable plots. 
[0089] The invention having been disclosed in connection with the foregoing 
preferred arrangements, variations will now be apparent, and should be considered 
encompassed within the scope and spirit of the invention. 
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